Are you sure you want to delete that group set? When you delete a group set, all of the groups within that set are also deleted! The references are not deleted; they still reside in your library.
To delete a group set:
Right-click on a group set heading to display a menu.
From the menu, select Delete Group Set.
You cannot delete any of the permanent group sets such as Online Search and Find Full Text.
You also cannot delete the last group set left in a library. Each library must have at least one group set as a place to store new custom groups, smart groups, and/or combination groups.